垂直分布式学习利用了多个学习工人收集的本地特征，以形成更好的全球模型。但是，工人与模型聚合器之间的数据交换进行参数培训会导致沉重的沟通负担，尤其是当学习系统建立在容量受限的无线网络上时。在本文中，我们提出了一个新型的层次分布式学习框架，每个工人分别学习了其本地观察到的数据的低维嵌入。然后，他们执行沟通有效的分布式最大 - 以有效地将合成的输入传输到聚合器。对于通过共享无线通道进行的数据交换，我们提出了一个基于机会性载体传感的协议，以实现所有学习工人的输出数据的最大功能操作。我们的仿真实验表明，提出的学习框架能够使用学习工人的所有原始输出的串联来实现与学习模型几乎相同的模型精度，同时需要独立于工人数量的沟通负载。

近年来，隐含的生成模型（例如生成对抗网络和扩散模型）已变得普遍。虽然这些模型确实显示出了显着的结果，但评估其性能是具有挑战性的。这个问题对于推动研究并从随机噪声中确定有意义的收益至关重要。当前，启发式指标（例如INCEPTION评分（IS）和特雷希特（Frechet Inception）距离（FID）是最常见的评估指标，但是它们所测量的内容尚不完全清楚。此外，关于他们的分数实际有多有意义的问题。在这项工作中，我们通过生成高质量的合成数据集来研究生成模型的评估指标，我们可以在该数据集中估算经典指标以进行比较。我们的研究表明，尽管FID和与几个F-Diverence确实相关，但它们的近距离模型的排名可能会差异很大，因此在用于Fain Graining比较时，它们有问题。我们进一步使用了这种实验环境来研究哪些评估度量与我们的概率指标相关。最后，我们研究用于FID等指标的基本功能。

在多任务学习（MTL）中，对联合模型进行了培训，可以同时对几个任务进行预测。联合培训降低了计算成本并提高数据效率；但是，由于这些不同任务的梯度可能需要冲突，因此训练MTL的联合模型通常比其相应的单任务对应人员产生的性能较低。减轻此问题的一种常见方法是使用特定的启发式方法将每个任务梯度组合到联合更新方向上。在本文中，我们建议将梯度组合步骤视为一个议价游戏，在该游戏中，任务就达成了有关参数更新联合方向的协议。在某些假设下，议价问题具有独特的解决方案，称为NASH讨价还价解决方案，我们建议将其用作多任务学习的原则方法。我们描述了一种新的MTL优化程序NASH-MTL，并为其收敛性得出了理论保证。从经验上讲，我们表明NASH-MTL在各个域中的多个MTL基准上实现了最新的结果。

通过随机梯度Langevin Dynamics（SGLD）的贝叶斯学习已被建议用于私人学习。尽管先前的研究在算法的初始步骤或接近融合时为SGLD提供了不同的隐私范围，但在两者之间可以解决哪些差异隐私保证的问题。这个临时区域非常重要，尤其是对于贝叶斯神经网络，因为很难保证与后部的融合。本文表明，即使从后验进行采样时，使用SGLD可能会导致此中期区域无限制的隐私损失。

Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics. The interplay of components can give rise to complex behavior, which can often be explained using a simple model of the system's constituent parts. In this work, we introduce the neural relational inference (NRI) model: an unsupervised model that learns to infer interactions while simultaneously learning the dynamics purely from observational data. Our model takes the form of a variational auto-encoder, in which the latent code represents the underlying interaction graph and the reconstruction is based on graph neural networks. In experiments on simulated physical systems, we show that our NRI model can accurately recover ground-truth interactions in an unsupervised manner. We further demonstrate that we can find an interpretable structure and predict complex dynamics in real motion capture and sports tracking data.

The United States coastline spans 95,471 miles; a distance that cannot be effectively patrolled or secured by manual human effort alone. Unmanned Aerial Vehicles (UAVs) equipped with infrared cameras and deep-learning based algorithms represent a more efficient alternative for identifying and segmenting objects of interest - namely, ships. However, standard approaches to training these algorithms require large-scale datasets of densely labeled infrared maritime images. Such datasets are not publicly available and manually annotating every pixel in a large-scale dataset would have an extreme labor cost. In this work we demonstrate that, in the context of segmenting ships in infrared imagery, weakly-supervising an algorithm with sparsely labeled data can drastically reduce data labeling costs with minimal impact on system performance. We apply weakly-supervised learning to an unlabeled dataset of 7055 infrared images sourced from the Naval Air Warfare Center Aircraft Division (NAWCAD). We find that by sparsely labeling only 32 points per image, weakly-supervised segmentation models can still effectively detect and segment ships, with a Jaccard score of up to 0.756.

We present a human-in-the-loop evaluation framework for fact-checking novel misinformation claims and identifying social media messages that violate relevant policies. Our approach extracts structured representations of check-worthy claims, which are aggregated and ranked for review. Stance classifiers are then used to identify tweets supporting novel misinformation claims, which are further reviewed to determine whether they violate relevant policies. To demonstrate the feasibility of our approach, we develop a baseline system based on modern NLP methods for human-in-the-loop fact-checking in the domain of COVID-19 treatments. Using our baseline system, we show that human fact-checkers can identify 124 tweets per hour that violate Twitter's policies on COVID-19 misinformation. We will make our code, data, and detailed annotation guidelines available to support the evaluation of human-in-the-loop systems that identify novel misinformation directly from raw user-generated content.

As language models (LMs) scale, they develop many novel behaviors, good and bad, exacerbating the need to evaluate how they behave. Prior work creates evaluations with crowdwork (which is time-consuming and expensive) or existing data sources (which are not always available). Here, we automatically generate evaluations with LMs. We explore approaches with varying amounts of human effort, from instructing LMs to write yes/no questions to making complex Winogender schemas with multiple stages of LM-based generation and filtering. Crowdworkers rate the examples as highly relevant and agree with 90-100% of labels, sometimes more so than corresponding human-written datasets. We generate 154 datasets and discover new cases of inverse scaling where LMs get worse with size. Larger LMs repeat back a dialog user's preferred answer ("sycophancy") and express greater desire to pursue concerning goals like resource acquisition and goal preservation. We also find some of the first examples of inverse scaling in RL from Human Feedback (RLHF), where more RLHF makes LMs worse. For example, RLHF makes LMs express stronger political views (on gun rights and immigration) and a greater desire to avoid shut down. Overall, LM-written evaluations are high-quality and let us quickly discover many novel LM behaviors.

Automatic defect detection for 3D printing processes, which shares many characteristics with change detection problems, is a vital step for quality control of 3D printed products. However, there are some critical challenges in the current state of practice. First, existing methods for computer vision-based process monitoring typically work well only under specific camera viewpoints and lighting situations, requiring expensive pre-processing, alignment, and camera setups. Second, many defect detection techniques are specific to pre-defined defect patterns and/or print schematics. In this work, we approach the automatic defect detection problem differently using a novel Semi-Siamese deep learning model that directly compares a reference schematic of the desired print and a camera image of the achieved print. The model then solves an image segmentation problem, identifying the locations of defects with respect to the reference frame. Unlike most change detection problems, our model is specially developed to handle images coming from different domains and is robust against perturbations in the imaging setup such as camera angle and illumination. Defect localization predictions were made in 2.75 seconds per layer using a standard MacBookPro, which is comparable to the typical tens of seconds or less for printing a single layer on an inkjet-based 3D printer, while achieving an F1-score of more than 0.9.

As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as 'Constitutional AI'. The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use 'RL from AI Feedback' (RLAIF). As a result we are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.